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We show that the empirical spectral distribution (ESD) of the sample autocovariance matrix 
(ACVM) converges as the dimension increases, when the time series is a linear process with 
reasonable restriction on the coefficients. The limit does not depend on the distribution of the 
underlying driving i.i.d. sequence and its support is unbounded. This limit does not coincide with 
the spectral distribution of the theoretical ACVM. However, it does so if we consider a suitably 
tapered version of the sample ACVM. For banded sample ACVM the limit has unbounded 
support as long as the number of non-zero diagonals in proportion to the dimension of the 
matrix is bounded away from zero. If this ratio tends to zero, then the limit exists and again 
coincides with the spectral distribution of the theoretical ACVM. Finally we also study the LSD 
of a naturally modified version of the ACVM which is not non-negative definite. 
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1. Introduction 

Let X ~ {Xt} be a stationary process with E(A"t) = and E(Xj'^) < oo. The autocovari- 
ance function (ACVF) 7x(') and the autocovariance matrix (ACVM) S]„(A') of order n 
are defined as: 

jx{k) = cov{Xo,Xk), fc = 0,l,... and E„(X) = ((7a'(i - j)))i<i,j<„. 

To every ACVF, there corresponds a unique distribution, called the spectral distribution, 
Fx{-) which satisfies satisfies 




for all h. 




We shall assume that 
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oo 

^|7x(fc)|<oo. (1.2) 

fc=i 

Then Fx{-) has a density, known as the spectral density of X or of jxi'), which equals 

oo 

fx{t)= exp{-2TTitk)jx{k), t£{0, 1]. (1.3) 

The non-negative definite estimate of T>n{X) is the sample ACVM 

n— \ k\ 

r„(X) = ((7jf(i - j)))i<ij<„ where ^x{k) = XiX^j^\k\. (1.4) 

i=l 

Under suitable assumptions on for every fixed fc, ^x{k) — > ^x{k) almost surely 

(a.s.). However, the largest eigenvalue of I],i(X) — r„(X) does not converge to zero, even 
under reasonable assumptions (see Wu and Pourahmadi (2009), Mcmurry and Politis 
(2010) and Xiao and Wu (2011)). Thus, one natural curiosity is what is the behavior 
of Tn{X) as n — )• ool We study this question through the behavior of its spectral dis- 
tribution. Suppose that Anxn is any real symmetric matrix. Let Ai, A2,..., A„ be its 
eigenvalues. The Empirical Spectral Distribution (ESD) of An is defined as, 

n 

F'4"(a;) = n-i^I(A, < x). (1.5) 

i=l 

Suppose converges weakly to F. Then we write i^^" ^ F. Suppose X is any 

random variable with distribution F . Then we say that X or is the Limiting Spectral 
Distribution (or measure) (LSD) oi F^^ . The entries of ^„ may be random. In that case, 
the limit is taken to be either in in probability or (as in this paper) in a.s. sense. 

Any matrix T„ of the form ((ti_j))i<i .,■<„ is a Toeplitz matrix and hence S„(X) 
and Tn{X) (with a triangular sequence of entries) are Toeplitz matrices. Suppose T„ is 
symmetric. Then from Szego's theory of Toeplitz operators (see Bottcher and Silberman 
(1998)), if X] l^fel < then the LSD of T„ equals f{U) where U is uniformly distributed 
on (0, 1] and f[x) — J^T^-oo^kCxp {—2TTixk), x g (0, 1]. In particular if (1.2) holds, 
then the LSD of S„(X) equals fx{U) where fx{-) is as defined in (1.3). 

We assume that {Xt} is a linear process 

00 

Xt=Y,dket-k (1.6) 

fe=0 

where {Ok} satisfies a weak condition and {et, t £ Z} is a sequence of independent 
random variables with appropriate conditions. The simulations of Sen (2006) suggested 
that its LSD exists and is independent of the distribution of {si} as long as they are 
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i.i.d. with mean zero and variance one. Basak (2009) and Sen (2010) initially studied 
respectively the special cases where X is an i.i.d. process or is an MA(1) process. 

Theorem 2.1 states that the LSD of r„(X) exists if {Xt} satisfies (1.6) and is universal 
when {£(} are independent with mean zero and variance 1 and are either uniformly 
bounded or identically distributed. The LSD is unbounded when > for all i. 

When {Xt\ is a finite order linear process, the limit moments can be written as 
multinomial type sums of the autocovariances (see expression (2.4)). When X is of infinite 
order, the limit moments are the limiting values of these multinomial expressions as the 
order tends to infinity. Some additional properties of the limit moments are given in the 
companion technical report Basak et al. (2011). 

Incidentally, r„(X) reminds us of the sample covariance matrix, 5*, whose spectral 
properties are well known. See Bai (1999) for some of the basic references. In particular 
the LSD of S (with i.i.d. entries) under suitable conditions is the Marcenko-Pastur law 
and is supported on the interval [0, 4]. Thus the LSD of r„(X) is in sharp contrast. 

The LSD of E„(X) depends on the parameters {9k} but there is no one to one corre- 
spondence between them. For instance, the LSD is same when X is AR(1) with parameter 
or —9. The same situation persists for the LSD of r„(X) (see Remark 2.2(iii)). 

A sequence of estimators {En} of I]„(X) may be called consistent if its LSD is fx{U). 
Thus {r„(X)} is inconsistent. It can be modified by tapering or banding to achieve 
consistency. For a sequence of integers m :— run — > oo, and a kernel function K{-) define 

m 

fx{t)= K{k/m)exp{-2mtk)jx{k), t e {0, 1] (1.7) 

k— — m 

as the kernel density estimate of fx{-)- Considering this as a spectral density, the corre- 
sponding ACVF is given by (for — m < h < m): 



m)^x{h). 



lK{h) = i eyip{2Trihx)fxix)dx 

i(0, 1] 

rn „ 

= y K{k/m) I ex'p{2TTihx — 2TTixk}^x{k)dx = K{h/ 
and is otherwise. This motivates the consideration of the tapered sample ACVM 

Tn^KiX) = {{K{{i - j)/m)^x{l - 3)))l<^,J<n. (1.8) 

If if is a non- negative definite function then Tn^K(,X) is also non- negative definite. 
Among other results, Xiao and Wu (2011) also showed that under the growth condi- 
tion m„ — o{n'') for a suitable 7 and suitable conditions on K, the largest eigenvalue 
of Tn,K{X) — Y,n{X) tends to zero a.s.. Theorem 2.3(c) states that under the minimal 
condition rrin/ri 0, ii K is bounded, symmetric and continuous at and K{0) ~ 1, 
then Tn,KiX) is consistent. This is a reflection of the fact that the consistency notion 
of Xiao and Wu (2011) in terms of the maximum eigenvalue is stronger than our notion 
and hence our consistency holds under weaker growth condition on to„. 
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The second approach is to use banding as in Mcmurry and Politis (2010) who used it to 
develop their bootstrap procedures. We study two such banded matrices. Let {mnlneN — >■ 
oo be such that a„ := m„/n — > a g [O7 !]• Then the Type I banded sample autocovariance 
matrix T^'^{X) is same as r„(X) except that we substitute for jxik) whenever |fc| > 
m„. This is the same as Tn^K with K{x) = /{|x|<i}- The Type II banded ACVM T'^'^^X) 
is the TUn X m„ principal sub matrix of r„(X). Theorem 2.3(a), (b) states our results 
on these banded ACVMs. In particular, the LSD exists for all a and is unbounded when 
a 7^ 0. When a = 0, the LSD is fx{U) and thus those estimate matrices are consistent. 

A related matrix, which may be of some interest, specially to probabilists, is, 

n 

r:(^) = ((7i(N-j|)))i<«j<n where ^*^{k) = n-^Y.^^^^+i^ ,fc = 0,l,.... (1.9) 

1=1 

r* (X) does not have a "data" interpretation unless one assumes we have 2n — 1 obser- 
vations Xi, . . . , X2n-i- It is not lion- negative definite and hence many of the techniques 
applied to r„(X) are not available for it. Theorem 2.2 states that its LSD also exists 
but under stricter conditions on {Xj}. Its moments dominate those of the LSD of r„(X) 
when di > for all i (see Theorem 2.2(c)) even though simulations show that the LSD 
of F* (X) has significant positive mass on the negative axis. 

2. Main results 

We shall assume that X = {Xt}tez is a linear (MA(oo)) process 

00 

Xt^Y.^kSt-k (2.1) 

where {£4, t G Z} is a sequence of independent random variables. A special case of this 
process is the so called MA((i) where Ok = for all k > d. We denote this process by 

= {Xt,d = Ooet + eiEt-i + ■■■ + OdEt-d, t e Z} {00 ^ 0). 

It may also be mentioned that working with two sided moving average entails no differ- 
ence. The conditions on {et} and on {9^} that will be used are: 

Assumption A. (a) {£4} are i.i.d. with E[£i] = and K[ef] = 1. 

(b) {et} are independent, uniformly bounded with E[et] = and E[ef ] = 1. 

Assumption B. (a) Oj > for all j. 

(b) Er=oi^.i<o«- 

The series in (2.1) converges a.s. under Assumptions A(a) (or A(b)) and B(b). Further, 
X and X^'^^ are strongly stationary and ergodic under Assumption A(a) and weakly 
(second order) stationary under Assumptions A(b) and B(b). 
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The ACVF of X^'^^ and X are given by 

d—j oo 

7x«<) (j) = ^kOj+k and 7x(j) = ^ ^fc^.+fc- (2.2) 

fc=0 fc=0 

Let {fci} stand for suitable integers and let 

k= (ko-.-kd), = {k : fco, . . . > 0, fco H \-kd^h}. (2.3) 

Theorem 2.1. (Sample ACVM) Suppose Assumption A(a) or A(b) holds. 

(a) Then a.s., F^"^'^'' F^ which is non-random and does not depend on the distri- 

bution of {et}. Further, for some sequence of constants {p^^}, 

hA = / x^'dFdix) = {{[ixi^A^P- (2.4) 

(h) Under Assumption B(b), a.s., F which is non-random and independent of 

the distribution of {et}. Further for every fixed h, as d — >■ oo, 

Fd^F and Ph,d^Ph= J x'' dF{x). 

(c) Under Assumption B(a), Fd has unbounded support and Ph,d-i ^ Ph,d if d > 1. 
Consequently, if Assumptions B(a) and B(b) hold, then F has unbounded support. 

Theorem 2.2. Suppose Assumption A(b) holds. Then conclusions of Theorem 2.1 con- 
tinue to hold for r* (X), d < oo, and (2.4) holds with some modified constants {p'^'^^}- 

Remark 2.1. (i) From the proofs, it will follow that the even limit moments {(32h,d} 

and {(32h} of the above LSDs are dominated by ■^-7n^(Sfe°=o l^fel)'^'' which are the even 
Gaussian moments. Hence the limit moments uniquely identify the LSDs. 

(ii) All the above LSDs have unbounded support while fx{U) has support contained in 
V~^°^Qo \lx{^)\, X^^cxj l7x(^)l]- Simulations show that the LSD ofT*^{X) has positive 
mass on the negative real axis. 

(Hi) Since F* (X) is not non-negative definite, the proof of Theorem 2.2 for d = oo is 
different from the proof of Theorem 2.1 and needs Assumption A(b). A detailed discussion 
on the different assumptions is given in Remark 3.1 at the end of the proofs. 

Theorem 2.3. (Banded and tapered sample ACVM) Suppose Assumption A[b) holds. 

(a) Let < a < 1. Then all the conclusions of Theorem 1 hold for r°'^(X('^'') and 
some modified constants {p^'^'^'*''} and {p^'^^'*''^''} respectively in (2.4). 
Same conclusions continue to hold also for d ~ oo. 
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(h) lfa = 0, and Assumption B(h) holds, the LSD ofr^-^{X) and r^'"(X) are fx{U). 
(a) and (b) remain true for V^'^^ {X^'''^) and T"'^^ {X) under Assumption A(a) . 

(c) Suppose Assumption B(b) holds. Let K be hounded, symmetric and continuous at 0, 
K{Q) ~ 1, K{x) = for |x| > 1. Suppose m„ — ?> oo such that TO„/rt — > 0. Then the LSD 
ofTn,K{X) is fx{U) for d<oo. 

Remark 2.2. (i) When K is non-negative definite. Theorem 2.3(c) holds under As- 
sumption A (a). 

(ii) Xiao and Wu (2011) show that under the assumption m„ = o(n'^) (for a suitable j) 
and other conditions, the maximum eigenvalue o/I]„(X) — r„(X) tends to zero. 

(Hi) Each of the LSDs above are identical for the combinations {9q, 9 1,62, . . .), {Oq, — 61,62, . . .) 
and {—60, 61, —62, . . .). See Basak et al. (2011 ) for a proof, based on properties of the limit 
moments. The LSDs fx{U) of E„(X) are identical for processes with autocovariances 
(70, 71, . . . , 7d) and (70, —71, • . • , {—lY^dj- The same is true of all the above LSDs. 




Figure 1. Kernel density estimates of tlie ESD, n = 1000; 100 realizations of: Tn{X), a = 1 (top left), 
a = 1/2 (bottom left) and r*(X) (top right); T„{X), o 0, (m. = 10) (dashed line) and E„(X) 
(dotted line) bottom right. The input sequence is X ~ AB{1), et ~ N{0, 1), 6 = 1/2. 

3. Proofs 

Szego's theorem (or its triangular version) for non-random Toeplitz matrices needs summa- 
bility (or square summability) of the entries and that is absent (in the a.s. sense) 
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for r„(X). As an answer to a question raised by Bai (1999), Bryc et al. (2006) and 
Hammond and Miller (2005) showed that for the random Toeplitz matrix r„^e = {{e\i-j\)) 
where {et} is i.i.d. with mean zero variance 1, the LSD exists and is universal (does not 
depend on the underlying distribution of £i). Rose and Sen (2008) showed that the LSD 
of Tn,x = ((X|i_j|)) exists when X satisfies (1.6) with some additional assumptions. 
However, none of the above two results are applicable to r„(X) due to the non-linear 
dependence of jx{k) on {Xt}. 

The only properties known for the LSD of r„.£ arc that it is symmetric, has unbounded 
support and a bounded density. The moments of the LSD of T„ x when Xt is as in (1.6), 
may be written in a nice form involving {9k} and the moments of the LSD of r„.£. 
Unfortunately, a similar expression eludes us for the LSD of r„(X), primarily due to the 
non-linear dependence of the autocovariances {^{k)} on the driving {et}. We are thus 
unable to provide any explicit or implicit description of the LSD. 

Our two main tools will be (i) the moment method to show convergence of distribution 
and (ii) the bounded Lipschitz metric to reduce the unbounded case to the bounded case 
and also to prove the results for the infinite order case from the finite order case. Suppose 
{An} is a sequence of n x n symmetric random matrices. Let f3h{An) be the /i*'' moment 
of its ESD. It has the following nice form: 

1 " 1 
/3,(A„) = -^A,'' = -Tr(A:j). 

Then the LSD of {A„} exists a.s. and is uniquely identified by its moments {jih} given 
below if the following three conditions hold: 

(CI) E[/3;) (yl„)] — > f3h for all h (convergence of the average ESD). 

(C2) En=i E WhiAn) - nf3h{An)]f < oo. 

(C3) {Ph} satisfies Carlcman's condition: J^hLi f^2h^^'^ ~ 

Let dsL denote the bounded Lipschitz metric on the space of probability measures on 
M, topologising the weak convergence of probability measures (see Dudley (2002)). The 
following lemma and its proof may be found in Bai (1999). 

Lemma 1. (a) Suppose A and B are n x n real symmetric matrices. Then 

4i(i^^, F^) < - Tr(A - Bf. (3.1) 
n 

(h) Suppose A and B are p x n real matrices. Let X — AA^ and Y = BB^ . Then 

4i(i^^, i^^) < A Tr(X + Y) Tr[{A - B)iA - Bf]. (3.2) 

When a = 1, then without loss of generality for asymptotic purposes, we assume that 
m„ = n. The full ACVM r„(X) may be visualised as the case with a = 1. When {Xt} is 
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a finite order moving average process with bounded {e*}, we use the method of moments 
to establish Theorem 2.1(a). The longest and hardest part of the proof is to verify (CI). 
We first develop a manageable expression for the moments of the ESD and then show 
that asymptotically only "matched" terms survive. These moments are then written as 
an iterated sum, where one summation is over finitely many terms (called "words"). Then 
we verify (CI) by showing that each one of these finitely many terms has a limit. The 
(Ibl metric is used to remove the boundedness assumption as well as to deal with the 
infinite order case. Easy modifications of these arguments yield the existence of the LSD 
when < a < 1 in Theorem 2.3(a) and (b). The proof of Theorem 2.2 is a byproduct of 
the arguments in the proof of Theorem 2.1. However, due to the matrix now not being 
non-negative definite, we impose Assumption A(b). The proof of Theorem 2.1(a) is given 
in details. All other proofs are sketched and details are available in Basak et al. (2011). 



3.1. Proof of Theorem 2.1 

The first step is to show that we may without loss of generality, assume that {et} are 
uniformly bounded so that we may use the moment method. The standard proof of the 
following Lemma may be found in Basak et al. (2011). For convenience, we will write 

r„(x('^)) = r„.d. 

Lemma 2. If for every {et} satisfying Assumption A(h), r„(A"(''') has the same LSD 
a.s., then this LSD continues to hold if {et} satisfies Assumption A(a). 

Thus from now on we assume that Assumption A(h) holds. Fix any arbitrary positive 
integer h and consider the h^^ moment. Then, 



1 " 

Yn,d = -{{Yl:f))r,j = l....,n, ^h.ere Y^f =^Xt,dXt+\i-j\.dl(t+\i-j\<n), 

t=i 

M^n,d) = }t.{vij-^, e 

l<7ro— 7r/^,7ri.,...,7r/^_i<n 
^ h n 

= X! [W{^^t3^dXt^ + \^^_^-^^ld\t,+\^,^^-^,\<n)) ■{'^■'i) 

l<-iTa,...,-iTh<n j=l tj=l 

To express the above in a neater and more amenable form, define 

t = {tl,...,th), TT = (tTq, . . . , 7r;i_l), 

A = |(t,7r) : 1 < ti, . . . ,tft,7ro, . . . ,7r,i_i < n,7r/i = TToj, 

a(t,7r) = {ti,...,th,ti + \TTo-TTl\,...,th+\TTh-l-TTh\), 

a = (ai,...,a2h) e {l,2,...,2n}2'', 

2h h 
= W_{Xaj,d) and Ia(t,7r) = ^^(tj +|7rj- 1 -tTj | <n) • 
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Then using (3.3) we can write the so called trace formula^ 

E[/3h(r„,<i)] = ^a(t,.)IIa(t,.)] • (3.4) 

3.1.1. Matching and negligibility of certain terms 

By independence of {et}, E[Xa(t.7r)] = if there is at least one component of the product 
that has no St common with any other component. Motivated by this, we introduce 
a notion of matching and show that certain higher order terms can be asymptotically 
neglected in (3.4). We say 

• a is d-matched (in short matched) if V i < 2h,3j ^ i such that ja^ — aj\ < d. When 

d = this means a.i = aj . 

• a is minimal d-matched (in short minimal matched) if there is a partition V of 
{!,..., 2/1}, 

{l,...,2h} = ui^,{ikJk}, ^k<Jk (3.5) 
such that {u-} are in ascending order and 

\ax - flyl < d-^ {x,v} ^ {«fc,Jfe} for some k. 

For example, for d = 1, /i = 3, (1, 2, 3, 4, 9, 10) is matched but not minimal matched and 
(1, 2, 5, 6, 9, 10) is both matched and minimal matched. 

Lemma 3. ■ a is matched hut not minimal matched } ~ 0{n^~^). 

Proof. Consider the graph with vertices {1, 2, . . . , 2h}. Vertices i and j have an edge 
if \ai — ttjl < d. Let fc = # connected components. Consider a typical a. Let Ij be the 
number of vertices in the j-th component. Since a is matched, Ij > 2 for all j and Ij > 2 
for at least one j. Hence 2h — ^ > ^/c. That implies k <h~ \. Also if i and j are 

in the same connected component then ja^ — aj\ < 2dh. Hence the number of a^'s such 
that i belongs to any given component is 0{n) and the result follows. □ 

Now we can rewrite (3.4) as 

E[^,(r„,,)] = J^E[^X,(t^.)Ia(t,.)] + -7^IE[E^a(t,.)Ia(t,.)] 
" 1 ^2 

+ ^a(t,x)Ia(t,.)] = Ti + T2 + Ts (say), 

" 3 

where the three summations are over (t,7r) e A such that a(t,7r) is respectively, (i) 
minimal matched, (ii) matched but not minimal matched and (iii) not matched. 

By mean zero assumption, = 0. Since X^'s are uniformly bounded, by Lemma 3, 
T2 < — for some constant C. So provided the limit exists. 
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lim ni3h{Tn.d)] = lim V X^^t..)Ut.-)]- (3.6) 

(t,7r)e^:a(t,7r) is 
minimal matched 

Hence, from now our focus will be only on minimal matched words. 
3.1.2. Verification of (CI) for Theorem 2.1(a) 

This is the hardest and lengthiest part of the proof. One can give a separate and easier 
proof for the case d = 0. However, the proof for general d and for d = are developed in 
parallel since this helps to relate the limits in the two cases. 

Our starting point is equation (3.6). We first define an equivalence relation on the set 
of minimal matched a = a(t,7r). This yields finitely many equivalence classes. Then we 
may write the sum in (3.6) as an iterated sum where the outer sum is over the equivalence 
classes. Then we show that for every fixed equivalence class, the inner sum has a limit. 

To define the equivalence relation, consider the collection of {2d+l)h symbols (letters) 

Wh = {wtd.---.w^,...,w\ : k = l,...,h}. 

Any minimal d matched a = (ai, . . . , a2h) induces a partition as given in (3.5). With this 
a, associate the word w = w[l]w[2] . . . w[2h] of length 2h where 

w[ik] = u>o, w[jk] = wi if ~ aj^^l, 1 <k < h. (3.7) 

As an example, consider d ~ = 3 and a = {ai, . . . ,0^) — (1,21,1,20,39,40). 
Then the unique partition of {1,2,..., 6} and the unique word associated with a are 
{{1, 3}, {2, 4}, {5, 6}} and [wqWqWqwIwqW^i] respectively. 

Note that corresponding to any fixed partition V ~ {{ik,jk}A < k < h}, there are 
several a associated with it and there are exactly {2d+ 1)'' words that can arise from it. 
For example with d = l,h = 2 consider the partition V = {{1, 2}, {3, 4}}. Then the nine 
words corresponding to V are wlwjwQWj where i,j = —1, 0, 1. 

By a slight abuse of notation we write w G T' if the partition corresponding to w is 
same as V. We will say that 

• w[x] matches with w[y] (say w[x] « u][y]) iff w[x] ~ wf and w[y] = wf, for some k, I, V . 

• w is d pair matched if it is induced by a minimal d matched a (so w[x\ matches with 
w[y\ iff Ifla, - ay\ < d). 

This induces an equivalence relation on all d minimal matched a and the equivalence 
classes can be indexed by d pair matched w. Given such a w, the corresponding equiva- 
lence class is given by 

U{w) = {{t,TT) e A: w[ik]=w^,w[jk]=wj' 4^ 

a(t,7r)i^ - a{t,TT)j^ = I and Ia(t,7r) = 1}. (3.8) 

Then we may rewrite (3.6) as (provided the second limit exists) 
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lim E[/3^(r„,d)] - V V lim ^-^ V E[X,(t,,)Ia(t..)]. (3.9) 

V weV (t,7r)en(tu) 

By using the autocovariance structure, we further simphfy the above as foUows. Let 

W{\^) = {w: i^{s:\w[i,]-w[js\\=i}^K z = 0, 1, . . . , d}. 

Using the definitions oi^xw{') and of Sh,d given in (2.3), we may rewrite (3.9) as (for 
any set Z , -^Z denotes the number of elements in Z) 

d 

hm E[/3„(r„,,)] =Y.Y. E 1™ ^#n(u;) ^^Yix^., (z)]^' (3.10) 

V Sh,d «)ePnW(k) i=0 

provided the following limit exists for every word w of length 2h. 

p^^^ = hm ^#n(u;). (3.11) 

To show that this limit exists, it is convenient to work with 11* (u;) D Yi{w) defined as 

n*(w) = {{t,TT) e A: w[ih]=w'^, w[jk]=w'^ ^ 

a(t,7r)ij, -a(t,7r)j, =Z and Ia(t,7r) = 1}. (3.12) 

By Lemma 3 we have for every w, n~''''+^-'#(n*(w) — Ii{w)) 0. Thus it is enough to 
show that lim„^oo ^h+i #n*(u;) exists. 

For a pair matched w, we divide its coordinates according to the position of the 
matches as follows. For 1 < i < j < h, let the sets 5*^ be defined as 

Si{w) = {i : w\i] Ri w[j]}, S2{w) = {j : w\i] w[j]} , 

Ssiw) = {i : w\i] K w[j + h]}, Siiw) ^ {j : w\i] ^ w[j + h]}, 
Ssiw) = {i : w[i + /i] w w[i + h]}, Sq{w) — {j : w[i + h] w w[i + h]}. 

Let E and G <Z E he defined as 

E = {ti, . . . ,<^,7ro, . . . , TT,,}, G = {U\i e Si{w) U Ssiw)} U {^o} U {^,|^ + he S^iw)}. 

Elements in G are the indices where any matched letter appears for the the first time 
and these will be called the generating vertices. G has (h + 1) elements say u", . . . , u^^^ 
and for simplicity we will write 

G=Un^ « , . . . , <+i) and Ml = {1,2,..., n}. 

Claim 1: Each element of E may be written as a linear expression, (say Ai) of the 
generating vertices that are all to the left of the element. 

Proof of Claim 1: Let the constants in the proposed linear expressions be {mj}. 
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(a) For those elements of E that are generating vertices, we take the constants as rrij = 
and the hnear combination is taken as the identity mapping so that 

for all i e Si{w) U S3{w), Ai = U, 

and for all i + h Cz 6*5 (w), Ai+h+i = t^i- 

(b) Using the relations between Si{w) and 6*2 (w) induced by w, wc can write 

for all j £ S2{w), tj = Aj + rij, 

for some Uj such that \nj \ < d and define rrij ~ rij for j G S2{w) and Aj = Ai. 

(c) Note that for every tt wc can write 

ki-i - TTil = &i(7ri_i - TTi) for some 6^6 {-1,1}. 
Consider the vector b = (61, 62, • ■ ■ , G {^1; I}''- It will be a valid choice if we have 

bi{ni-i — TTi) > for aU i. (3.13) 
We then have the following two cases: 

Case 1: w[i\ matches with w[j + h], j + h € S'4(w) and i € 83(10). Then we get 

tj = tj + &j(7rj_i — TTj) + TLj+h for some integer rij+h S {~d, . . . , . . . , d\. (3.14) 

Case 2: w[i + h] matches with w[j + /)], j + /i G Sq{w) and i + /i G Sz{w). Then we have 
ti + \'Ki-i - 7r,| = tj + \'Kj-i - vTjl + Uj+h where e {-d, . . . , 0, . . . , d}. (3.15) 

So we note that inductively from left to right we can write 

^ -^j+i+h + "^j+i+h, j + h e Siiw) U Se{w). (3.16) 

Hence, inductively, TTj as a linear combination {Aj"} of the generating vertices up to an 
appropriate constant. The superscript b emphasizes that {Aj"} depends on b. Further, 
{Aj*} depends only on the vertices present to the left of it. □ 
Now we are almost ready to write down an expression for the limit. If Ai were unique 
for each b, then we could write #11* (w) as a sum of all possible choices of b and we 
could tackle the expression for each b separately. However, Ai's may be same for several 
choices bi G {—1, 1}- For example, for the word WqWqWqWq, we may choose any b. We 
circumvent this problem as follows: Let 

r = {.] + h e ^4(^^) U SeH I A]Vh - A]Vh_i ^ V b,}. 

Note that the definition of T depends on w only through the partition V it generates. 
Suppose j + h & Define 

imsart-bj ver. 2011/12/06 file: autocovrevbern.tex date: April 19, 2012 



Autocovariance matrix 



13 



mUn) := 6,(A]Vh-i(t/")-A]Vh((^"))+m,+„_i-7n,+, (3.17) 

:= Lj{Un) + rrij+h-i - nij+h- (3.18) 
Then from (3.14) and (3.15) the region given by (3.13) is 

{Lj{Un) > 0} = {Lj{Un) + m,+h-i - m,+h > 0}. (3.19) 

Claim 2: The above expression is same for all ehoices of {bj}, for j + h G T- 
Proof of Claim 2: First we show that ii j + h E T then we must have 

tj ~ tj + Ittj^i — TTjl + Tij for some integer \nj\ < d. (3.20) 

Suppose this is not true. So first assume that j + h G Se{w). Then we will have a relation 

ti + bi{'Ki^i — TTi) = tj + bj{TTj^i — TTj) + Uj, where i + h £ Sq{w). (3-21) 

Since Aj" depends only on the vertices present to the left of it, in (3.21), coefficient of tt^ 

would be non-zero and hence we must have AjY^-i ^ -^j+h ^ ^■ 

Now assume j + h € S^iw) and matches with w[j + for i ^ j . Then we can repeat 
the argument above to arrive at a similar contradiction. This shows that if j + /i £ T then 
our relation must be like (3.20). Now a simple calculation shows that for such relations, 

6i(A]Vh-i(C^«) - A]Vh(t^")) + ruj+h-i - mj+h = -nj 

which is of course same across all choices of b. This proves our claim. □ 
Now note that ii j + h G T and if nj+h ^ then as we change bj it does change the 

value of 7712^+1- Further, we can have at most two choices for tt^ for every choices of TTj-i 

if rij+h 7^ depending on bj. 

However for j + /i G T and rij = we have only one choice for TTj given the choice for 

iTj-i for every choice of bj. On the other hand we know b G {—1, 1}'' must satisfy (3.13). 

Considering all this, let 

B{w) = {b e {-1, 1}'' I bj = l if nj = for j e T} 
where {rij} is as in Claim 2. For ease of writing we introduce a few more notation: 

^niJiiUn) ■■= K>^^2h+l(Un) + ^2/1+1 = h+l{Un) + m/^+i), 
h 2h 

and IriUn) := I(ij(C/n) > 0) x [| I(n, < 0). (3.22) 

Now we note that, 
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pi'^ := lini^#n*(«;) 

bei3{w) ;7„eAA,^+i 



lim > 

b6B(m 



Now it only remains to identify the limit. To this end first fix a partition V and b G 
{-1, 1}''. If d = 0, then there is one and only one word corresponding to it. However, 
across any d and any fixed ko,ki, . . . ^k^, the linear functions Aj's continue to remain 
same. The only possible changes will be in the values of mj's. 
We now identify the cases where the above limit is zero. 

Claim: Suppose w is such that TZ := ^^2h+iiUn) + fn2h+i = ^+\{Un) + mu+i^ is a 
lower dimensional subset of J^n~^^- Then the above limit is zero. 

Proof: First consider the case d = 0. Then rrij = 0, V j. Note that TZ lies in a hyper- 
cube. Hence the result follows by convergence of the Riemann sum to the corresponding 
Riemann integral. For any general d the corresponding region is just a translate of the 
region considered for mj = 0. Hence the result follows. □ 
Hence for a fixed w £ V, a positive limit contribution is possible only when TZ ~ ■N'n'^^ ■ 
This implies that we must have 

AWi(C^n)-^h+i(C^") ^ (for d = 0) 

^2h+i(t^n) ^ ^h+i(^n) and m2h+i - mh+i = (for general d). 

Note that the first relation depends only the partition T' but the second relation is 
determined by the word w. Now Aj" being linear forms with integer coefficients 

A]'(C/„) + m, G {1, . . . ,n} ^ A]' + e (0, 1]. 

Define Im,h{U), h^,L(U), h^{U) and lr{U) as in (3.22) with C/„ replaced by [/, L 
replaced by L, Afn replaced by (0,1), n replaced by 1, and dropping nij's in lxb „^. 
Noting — ^ U following uniform distribution on [0, 1]'*"''^, h+i lim#H*(z/;) equals 



= J2 K^AU) X h.,iiU) X I^.iU) X iriU) . (3.23) 



beis{w) 

Now the verification of (CI) is complete by observing that (3.10) becomes 

Jim E[/3,(r„,,)] = E E P^''I[^"fxi^>('^^'' = E ^^L'^ri[7x(^)«]' 
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where 

■!i>e-pnw(k) V 



'^''^= E ^EPk-'- (3.24) 



Verification of (C2) and (C3) for Theorem 2.1(a) 

Lemma 4. ^ E Trlr^^^,) - n-iE[Tr(r:j^,)]] ' = O (n-^) . /fence i Tr(r,^, ,) con- 
verges to Ph.d o,.s.. 

(b) {l3h,d}h>o satisfies (C3) and hence defines a unique probability distribution on R. 

Proof. Proof of part (a) uses ideas from Bryc et al. (2006) but the inputs of the matrix 
are no longer independent, and therefore some modifications are needed. Details are 
available in Basak et al. (2011). 

(b) Using (3.24) and (2.4) and noting that the number of ways of choosing the partition 
{l,...,2h} ~ uf^]^{i(, j;} for a(t,7r) is it easily follows that 

,^ . ^4'>(2/i)! h\ ,.,,,„ 

j=0 k=0 ' k=Q 

This implies (C3) holds, proving the lemma. Proof of Theorem 2.1(a) is now complete. □ 
3.1. 4-. Proof of Theorem 2.1(b) (infinite order case) 

First we assume {et} is i.i.d.. Fix £ > 0. Choose d such that X]fe>d+i l^fcl < £■ For 
convenience we will write r„(X) — r„. Clearly, r„ = AnA"^ where 

(^n)i,i = Xj^i, if 1 < j - i < n 
= 0, otherwise. 

By ergodic theorem, a.s., we have the following two relations: 

1 



i[Tr(r„.rf + r„)] 



2 
t 

t=l 



E[Xl, + X^]<2j2d 



oo 

2 
k- 

k=0 



iTr[(A„,,-^„)(A„,rf-A„)^] - iE(^*.'' - ^*)' l^t^M - ^t]' < E 

t=l k=d+l 

Hence using Lemma 1(b), a.s. 

oo 

limsup dlr.{F^"^-,F^") < 2(El^fc|)'e'- (3.26) 

fe=0 
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Now F'-"'" ^ Fd a.s.. Since dsL metrizes weak convergence of probability measures, as 
n — > oo, dBLiF^"^ "^ ,Fd) — > 0, a.s.. Since {-F''"" ''},i>i is Cauchy with respect to (Ibl 
a.s., by triangle inequality, and (3.26), limsup„ „ dBL{F^" , F^"") < 2\/2(X]fc°.o l^fel)^- 
Hence {-F''""}ti>i is Cauchy with respect to dsL a.s.. Since dsL is complete, there exists 
a probability measure F on M such that F^" —> F a.s.. Further 

oo 

dBL{Fd,F) ^\imdBL{F^--',F^") < V2(^|0fe|)e, 

k=0 

and hence Fd F as d ^ oo. Since {Fd} are non-random, F is also non-random. 

Now if {et} is not i.i.d. but independent and uniformly bounded by some C > then 
the above proof is even simpler. We omit the details. 

To show convergence of {f3h,d}, we note that under Assumption B(b), (3.25) yields 

sup \hA < Ch := ^J^(E l^fel)''' < V /i > 0. (3.27) 
Hence for every fixed h, {A'^} is uniformly integrable where Ad ^ Fd- Since Fd F, 
Ph= I x"" dF^ hm / x'' dFd = lim Ph.d, 

J ^ J d^oc 

completing the proof of (b). Since |/3/i| < Ch, it easily follows that {/3h}h>Q satisfies (C3) 
and hence uniquely determines the distribution F. □ 

3.1.5. Proof of Theorem 2.1(c) 

We first claim that for d > p^^^ ^.^ = Pk^^\i o- ih.\s, consider a graph G with 

2h vertices with h connected components and two vertices in each component. Let 

= {a : a is minimal d matched, induces G and la^; — aj^| = d + 1 
for some x, y belonging to distinct components of G}. 

Then one can easily argue that = 0{nP'~'^) and consequently #{(t, tt) e A\ a(t, tt) G 
M} = Gin''). Hence 

(d) 

Pko,...,ka ^ 

= lim — r— 7#s (t, 7r) e ^ I a(t, tt) is minimal d matched with partition {1, 2h} = U?, {H,ji} 

and there are exactly fc^many Vs for which 

|a(t,7r)(i;) - a(t,7r)(j;)| = s, s = 0, ...,d, Ia(t,7r) = 1 and 

|a(t, tt){x) — a(t, 7r)(2/)| > d + 2 if X, y belong to 

different partition blocks | 
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Thus for 6*0, . . . , 6*^ > and d > 1, 

Sh,d-l i=0 Sh,d~l i=0 

proving the resuh. 

IncidcntaUy, if Assumption B(a) is violated, then the ordering need not hold. This can 
be checked by considering an MA(2)and an MA(1) process with parameters 6*0, 6*1, ^2 
and where 82 = —k9o, do, ^1 > 0. Then (3-2^2 < ^2,1 if we choose k > sufficiently small. 
The details are available in Basak et al. (2011). 

3.1.6. Proof of unbounded support of Fd and F 

For any word w, let denote the length of the word. Let 

W = {w = W1W2 '. {wil = 2/1 = |u'2|; w, Wi, W2 are zero pair matched; Wi[x] matches with 
wi[y] iff 'W2[x] matches with [j/]}- 

Then 

I32h,d > [7x(^)(0)]'V>,o > [7x(^)(0)]"' y limn-(2''+i^#n*(u;). (3.28) 

For w = W1W2 S W, let {1, . . . , 2h} ~ ^i=iiis, js) be the partition corresponding to 
wi. Then 

lim„ ^^Lrr- > lim„ ;^3^#{(t, tt) : t,^ = tj^ and vr^, - tt^^^i = ttj^^i - -kj^ for 1 < s < /i; 

*j + ^ ■^j-il ^ for 1 < j < 2/1}. 

Now adapting the ideas of Bryc et al. (2006), we obtain that for each d finite Fd has 
unbounded support . Since {/3/i,d} increases to same conclusion is true for F. For 
details sec Basak et al. (2011). 

3.2. Outline of the proof of Theorem 2.3 

3.2.1. Proof of Theorem 2.3(a), (h) for the case < a < 1 

Let /3/i(r^'^) and /3/i(r"'^^) be the /i*'* moments respectively of the ESD of Type I and 
Type II ACVMs with parameter a. We begin by noting that the expression for these con- 

h h 

tain an extra indicator term Ii = J^I(|7ri_i — TTi\ < rUn) and I2 ~ 'W^i^ — — "^") 

respectively. For Type II ACVMs since there are m„ eigenvalues instead of n, the nor- 
malising denominator is now ?7i„. Hence 



^ h n 



-ll<n: 



l<7ro,...,7rh<n j = l tj=l 
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and 

_^ h n 

l<7ro,...,7rh<)i j=l tj = l 

It is thus enough to estabhsh the hmits on the right side of the above expressions, and 
we can follow similar steps as in the proof of Theorem 2.1. 

Since there are only some extra indicator terms, the negligibility of higher order edges 
and verification of (C2) and (C3) needs no new arguments. Likewise, verification of (CI) 
is also similar except that there is now an extra indicator term in the expression for 
Pw'^. This takes care of the finite d case. For d — oo, note that the Type II ACVMs are 
TUn X m„ principal subminor of the original sample ACVMs and hence are automatically 
non-negative definite. We can write T"'^^ {X^'^^) — {A'^'^J){A"'^/)^ where A"'^J is the 
first TO„ rows of An^d- Thus imitating the proof of Theorem 2.1. we can move from 
finite d to d = oo. However for Type I ACVMs we cannot apply these arguments, as 
these matrices may not be non-negative definite. Rather we proceed as in the proof of 
Theorem 2.2. Previous proof of unbounded support now needs only minor changes. We 
omit the details. 

Since T"'^/ is non-negative definite, the technique of proof of Theorem 2.1 may be adopted 
under Assumption A(a). □ 



3.2.2. Proof of Theorem 2.3(h) for Type I hand ACVM 

Existence: Let plf"*'*^'^ be the limiting contribution of the word w for Type I ACVM 
with band parameter a = 0. Then 

pWA/ lim^ V Ec/„ k„,;,([/„) X I^i, ,„([/„) X I;,!, i([/„) X 



where 



h 



If for word w, Aj^h-i 7^ '^j+h some j then I-r,L{Un) — >■ as n — > cxd and thus limiting 
contribution from that word will be 0. Thus only those words w for which A[j_[_j^ = Aj^h 
for all j G {1, 2, . . . , /i + 1} may contribute non-zero quantity in the limit. This condition 
also implies that, for such words no tt^ belongs to the generating set except ttq. This 
observation together with Lemma 6 of Basak et al. (2011), and the expression for limiting 
moments for Tn{X) shows that w €! Wq may contribute non-zero quantity, where 

Wq ~ {w : \w\ — 2h, w[i] matches with w\i + h], rii < 0, i = 1, 2, . . . , h}. 

Further note that ii w e Wq then T = {h + 1, h + 2, . . . , 2h}, and thus Ir,L = 1- 
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For d = note that #Wq = 1 for every h and one can easily check that the contribution 

from that word is 1. Thus /3° g = 0^^ and as a consequence, the LSD is Sgi. 

Now let us consider any < d < oo. Note that for any d finite, and if m„ > d, then 

h 

I;^b ,„ X Ix^.L ^ Ir,m " ^ W ^{t^J ^ 0), aS n — )• OO. 

Combining the above arguments wc get that for any w e Wq , Pw^''^'^ is the number of 
choices of b G B{w), and {ni, 712, . . . ,nh] rn < 0}, such that riib; = 0. 

Noting that Type I ACVMs are not necessarily non- negative definite, we need to adapt 
the proof of Theorem 2.2. Details are omitted. 

Identification of the LSD: Now it remains to argue that the limit we obtained is same 
as fx{U). For d = LSD is 602 and it is trivial to check it is same as fx{U). 
For < (i < 00, note that the proof does not use the fact that m„ — >■ 00 and we further 
note that for any sequence {m„} the limit we obtained above will be same whenever 
liminf„_^oo "nin > d. So in particular the limit will be same if we choose another sequence 
{mjj} such that = d for all n. Let F^, ^ denote the Type I ACVM where we put 
instead of ^xw{k) whenever k > m'^ and let I]„ c; be the n x n matrix whose {i,jY^ 
entry is the population autocovariance Jxw (I* ~ j Now from Lemma 1(a) we get 

< 2fe(., (0) - 7x(.) (0))2 + • • • + 2(7;^ {d) ~ 7^Y«.) {d)f. 

For any j as n — )- 00, 7x(<i) 0) Ixw (j) a.s.. Since d is finite, the right side of the above 
expression goes to a.s.. This proves the claim for d finite. 

To prove the result for the case d = 00, first note that we already have 

LSD(r°^l^) - LSD{^r.M) Ga and LSD{T"^'^) ^ LSDiT"!;') as d ^ 00. 

Thus it is enough to prove that Gd —> G(= LS* £>(!]„)) as d — 00 where E„ is the nx n 
matrix whose {i,jY''' entry is 7x(|* — Define a sequence of n x 71 matrices f^n.d whose 
(ijj)*'* entry is 7x(|« — j\) if |* ^ j\ ^ d and otherwise 0. By triangle inequality. 

Fix any e > 0. Fix do such that ^S^o l^il) (Sz^d+i l^'l) < for aU d > dg. Now 
again using Lemma 1(a) we get the following two relations: 

limsup4i(F^"•^F^"-) < 2[(7;f<.,(0) - 7^(0))^ + • • • + hxi^>{d) - rx{d))'] , 

d 00 2 2 

= 2E( E ^16' 

3=0 k=d-j+l 

1 - 

^2^ (^s„,,^^s„^ < limsup-Tr(E„,d-E„)2 < — . 

n n lb 
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Thus \imsup dBLiF^"''' , F'^") < e/2, for any d > da, and therefore by triangle mequahty, 

n 

dsLiF'^^F^) < e. This completes the proof. □ 

3.2.3. Proof of Theorem 2.3(h) for Type II band autocovariance matrix 

First note that by Lemma 3 we need to consider only minimal matched terms. Let 

Gt = {U ■■ U e G} and G„ = {tt,; : tt,; e G}. 

Since 1 < tt^ < m„ for all i, by similar arguments as in Lemma 3 we get 

number of choices of a(t,7r) = 0(n^'^*mf''^"). 

Thus for any word w such that 4/^Gt < h the limiting contribution will be 0. Hence only 
contributing words e in this case are those for which ^83(10) = #5*4 (w) = h. and from 
Lemma 6 of Basak et al. (2011), the only contributing words are those belonging to Wq. 
Therefore using same arguments as in the proof of Theorem 2.3, for Type I ACVM, for 
a = we obtain the same limit. All the remaining conclusions here follow from the proof 
for Type I ACVMs with parameter a ~ 0. 

Since Type II ACVMs are non-negative definite, connection between the LSD for finite 
d and d = 00 is proved adapting the ideas from the proof of Theorem 2.1. □ 

3.2.4. Proof of Theorem 2.3(c) 

Since K is bounded, negligibility of higher order edges and verification of (C2) and (C3) 
is same as before. Verification of (CI) is also same, with an extra indicator in the limiting 
expression. Denoting pw^'^ to be the limiting contribution from a word w, we have, 

I™.h(C/n) X IA^™(C/„) X I;,b_i(C/„) X l\{Un) X lif (i7„)] , 

where 

iK(f/„) :=nx(^:^). 

Since m„ — > 00, and K{-) is continuous at 0, K{Qi) = 1, note that 1^ — > 1. Now arguing 
as in Section 3.2.2, we get pl^"*'*^'^ = Pw^'^ for every word w and thus the limiting 
distributions are same in both the cases. For the case d = 00 the arguments arc similar 
as in Section 3.2.2 and the details are omitted. □ 



pW'^ = limEt,, 



3.3. Proof of Theorem 2.2 



Proceeding as earlier it is easy to see the limit exists, and for each word w, the limiting 
contribution is given by, 



■Xd) 



I„u,{U)xI^^{U)xIr{U) 
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Comparing the above expression with the corresponding expression for the sequence Tn,d, 

pKd < Pld if 0, >0,0<j<d. 

Relation (3.25) holds with /3h.d replaced by ^j. We can use this to prove tightness of 
{F^} under Assumption B(a) and thus also Carleman's condition is satisfied. 
Since F* and F* ^ are no longer positive definite matrices the ideas used in the proof of 
Theorem 1(b) cannot be adapted here. We proceed as follows instead: Note that 

^ h h 



Write 



71" 

Xt^ = E ^k,et^-k, and Xt^+\^^_^_^^\ = ^ ^k'^^t,+\^,_,-^,\-k]- 

kj>0 k'j>0 

Then using the absolute summability Assumption B(b) and applying DCT we get 

h 

,■— 7r^_i I — fc . 



n" 

fc,-,^>0J = l (t.n)eA 3 = 1 



Using the fact that {etjj^i are uniformly bounded and absolute summability of {^fej^i 
we note that it is enough to show that the limit below exists. 

ii^n-(''+%[ E n(^*.-.s+k,^^.-.i-.;) 

(t.7r)e.A j=l 

One can proceed as in the proof of Theorem 2.1 to show that only pair matched words 
contribute and hence enough to argue that 

limn~(''+^)#{(t, tt) e A : {tj^-kj, tj + \nj—Trj^i\ — kj, j — l,...,h} is pair matched} 
exists, and which follows by adapting the ideas used in the proof of Theorem 2.1. Note 
that some compatibility is needed among {kj,kj, j = l,...,/i}, the word w and the 
signs bi (= ±1) to ensure that the condition tto = tth is satisfied. So the above limit will 
depend on {fcj, fc^, j ~ 1, . . . ,h}. 

We also note that 



1™-^ E e ■ {^3 - ^J' + \'^3 ~ '^3-l\ - = £ n(w)} 

w pair matched, 

\w\=2h 

4^(2/i)! 
- ~~hl ■ 

Hence F* is uniquely determined by its moments and using DCT, 13^ ^ ^ P^- Whence it 
also follows that F^ A F*. Proof of part (c) is similar to the proof of Theorem 2.1(c). □ 
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Remark 3.1. Theorem 2.2 has not been proved under Assumption A(a) because there 
is no straightforward way to apply (3.1) or (3.2) since F* (X) is not non-negative definite. 
Simulation results indicate that the same LSD continues to hold under Assumption A (a). 
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